Reliability of unattended ground sensors (UGS) to detect and classify different activities (e.g., walking and digging) is often limited by high false alarm rates, possibly due to the lack of robustness of the underlying algorithms in different environmental conditions (e.g., soil types and moisture contents for seismic sensors), inability to model large variations in the signature of a single activity and limitations of on-board computation. Tactical scenarios, pertinent to border control and surveillance, are richly equipped with multi-modal sensing devices (e.g., acoustic, seismic, passive infrared, and magnetic), referred to as unattended ground sensors (UGS). Such systems are deployed to detect and classify different types of targets and activities in real time, which requires a holistic situation awareness. Despite the high false alarm rates, the UGS systems are preferred because they are relatively inexpensive, easy to deploy and unobtrusive to the surroundings. The high false alarm rates may be attributed to inadequate on-board processing algorithms and the lack of robustness of the detection algorithms in different environmental conditions (e.g., soil types and moisture contents for seismic sensors). Furthermore, limited battery operating life have made power consumption a critical concern for both sensing and information communication.
Seismic sensors have performed with the highest reliability compared to other components of UGS systems regarding target detection and activity classification because they are less sensitive to Doppler effects and environment variations as compared to acoustic sensors. Present personnel detection methods using seismic signals may be classified into three categories, namely, time domain methods, frequency domain methods, and time-frequency domain methods. More recently, feature extraction from (wavelet-transformed) time series, based on symbolic dynamic filtering (SDF), has been proposed by X. Jin, S. Sarkar, A. Ray, S. Gupta, and T. Damarla, “Target detection and classification using seismic and PIR sensors,” IEEE Sensors Journal, vol. 12, pp. 1709-1718, June 2012 (herein incorporated by reference) for target detection and classification in border regions. The rationale for using wavelet-based methods is denoising and time-frequency localization of the underlying sensor time series. However, this method requires tedious selection and tuning of several parameters (e.g., wavelet basis function and scales) for signal pre-processing in addition to the size of the symbol alphabet that is needed for SDF. In S. Bahrampour, A. Ray, S. Sarkar, T. Damarla, and N. M. Nasrabadi, “Performance comparison of feature extraction algorithms for target detection and classification,” Pattern Recognition Letters, vol. 34, pp. 2126-2134, (December 2013) (herein incorporated by reference), a comparison shows consistently superior performance of SDF-based feature extraction over Cepstrum-based and PCA-based feature extraction, in terms of successful detection, false alarm, and misclassification rates, using data collected for border-crossing detection. The reliability of the performance by SDF, in varied environmental conditions for personnel detection problem, was studied in N. Virani, S. Marcks, S. Sarkar, K. Mukherjee, A. Ray, and S. Phoha, “Dynamic data driven sensor array fusion for target detection and classification,” Procedia Computer Science, vol. 18, pp. 2046-2055 (December 2013), herein incorporated by reference.
There has been numerous research on human activity recognition from data collected by wearable sensors (e.g., accelerometer), ubiquitous sensor net (e.g., passive infrared (PIR) sensor net), imaging and video sensors (e.g., wireless camera network). However, there has been relatively little work done in activity recognition based on the data collected by UGS, especially seismic sensor. The main challenge lies in the inherent multi-timescale nature, low SNR and high variability (different external conditions) of the seismic data for same class of activity.
It appears that there has been relatively little work done in activity recognition based on the data collected by UGS, especially seismic sensors. The present invention is directed to a system designed to detect and classify different human activities from seismic signatures in real time. One of the most significant and dreaded threat scenarios in tactical situations is comprised of the activities such as, a personnel walking to a site and digging there to bury explosives and walking away. It is challenging to detect and segment such activities from only seismic signatures in real time because of their inherent multi-timescale nature with low signal-to-noise-ratio (SNR) in varied environmental conditions. Also, the persistence level and type of digging activity have a significant variability, which make the problem more complex. In seismic signals, both of these activities may appear as arrays of near-identical impulses at a fast time scale. But, it is the time evolution of those impulses in a slower time scale, which capture the separability of those activities.
There are several techniques proposed in the literature to determine gait of a person and classify whether the observed signature belongs to a human or an animal. In K. Houston and D. McGaffigan, “Spectrum analysis techniques for personnel detection using seismic sensors,” in Unattended Ground Sensor Technologies & Applications V, vol. 5090, pp. 162-173, SPIE (2003), the seismic signatures are analyzed in Fourier domain to look for the fundamental and harmonics of gait frequency. Since the gait of a person walking is different from that of a quadruped, the fundamental and harmonics frequencies for a person walking are different from those of a quadruped and thereby distinguishing a person or a quadruped walking. In H. Park, et al., “Cadence analysis of temporal gait patterns for seismic discrimination between human and quadruped footsteps,” in IEEE Conference on Acoustics, Speech and Signal Processing, pp. 1749-1752, (2009), the cadence analysis is done to extract temporal gait pattern which provides information on temporal distribution of the gait beats. However, these techniques result in a high number of false alarms or miss classification resulting in wasting human resources for investigation. Moreover, these techniques may or may not work in different soil conditions as the propagation properties of various soils are different, rendering the spectral based analysis prone to misdiagnosis. The reliability of the detection performance by SDF, in significantly varied environmental conditions for personnel detection problem, was studied in N. Virani, S. Marcks, S. Sarkar, K. Mukherjee, A. Ray, and S. Phoha, “Dynamic data driven sensor array fusion for target detection and classification,” Procedia Computer Science, vol. 18, pp. 2046-2055 (December 2013).
In U.S. Published Application No. 2008/0309482 (482), Honeywell Corp. implemented a tunnel activity detection. In the '482 patent application, several seismic sensors are deployed in the area of interest. If there is an underground activity, the sensors record the changes in the voltages and transmit them to a “tower” (paragraph [0036]), where it appears that a person determines if there is some activity in the ground by observing the changes in the voltage levels.
U.S. Pat. No. 7,656,288 to Joslin uses multiple sensors of different modalities to detect and classify an event. The event could be a person walking, vehicle traveling, etc.
In U.S. Published Application No. 2008/0109091 to Joslin, discloses a method for improved data communication with a remote sensor and communicating the data when a rule is satisfied. U.S. Pat. No. 7,714,714 to Volgewede, et al., discloses a system for improved signal processing using a remote sensor comprising a detection component and a classification component. The classification of an event is based at least in part on a situation. U.S. Pat. No. 7,710,265 to Volgewede, et al., discloses multiple sensors of different modalities to detect and classify an event such as a pedestrian walking, vehicle moving, etc., based on a set of rules and the rules are selected based at least in part on a situation.